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Abstract 

Native American programs have affective and cognitive objectives stated 
in project proposals-. Most evaluations focus upon the cognitive, 
quantitative results of particular projects, programs or interventions. 
Project evaluators need to use or develop instruments designed to assess 
the affective domain. This paper addresses the technical, psychometric 
pift)perties of attitude scale instruments. Specific attention was given 
to the history of attitude measurement, characteristics of attitudes, 
steps for scale construction, reliability, validity and other technical 
considerations. Appendices included an example development effort for 
evaluation of Indian Education projects. 
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Introduction 

Tribal agencies, school districts, local education agencies and 
many other organizations and groups must fulfill program evaluation 
requirements for final reports to sponsoring agencies. Program evalu- 
ations are designed to answer specific questions related to the results 
of a particular project, program, or intervention strategy. In the 
development of a handbook for evaluating Indian Education projects, 
William Demmert (1976) commented that one of the goals of evaluation is 
to "...provide Congress and the- U.S. Office of Education with an accurate 
understanding of the overall efficacy of Title IV projects in meeting 
the needs of Native Americans." The purpose of evaluation is to ^assess 
the indices of quality of a particular intervention or project. Usually 
the indices of quality refer back to the specific objectives of the 
project. Unfortunately J many evaluations overlook unintended outcomes 
or serendipitous results. 

It is common knowledge that most educational projects have attitu- 
dinal effects as a central concern; however, overemphasis upon quantita- 
tive evaluation plans shifted those concerns to more objective foci. 
There is no reason why a local education agency must ignore the attitu- 
dinal effects in the evaluation plan. In many instances the attitudinal 
or affective outcomes are tantamount or transcent the cognitive more 
quantitative aspects of the project. The evaluation literature is 
replete with final reports documenting "significance" or "non-significance 
in the quantitative arena and mere statements of conjecture alluding to 
the value of program "x" in changing participants opinions, attitudes, 
•or values. There is no reason for this practice to continuel Evaluation 
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research has matured enough to provide direct, measurable and quantifi- 
able indices of affective outcomes. Local education agencies must de- 
mand that evaluators address the affective as well as t?.- > "^itive 
effects of specific programs. 

Ralph Tyler (1973) put the problem in perspectiv ^ - "Evalu- 
ation projects are also criticized for their failure to af. the 
impact of a course or program in terms of the affective devo; : ^ent of 
students." Studies by Josephina, (1959); Allen, (1960); Nea^e. Gill, 
and Tismer, (1970); Woolley and Patalino, (1970); and Callahan, (1973) 
have illustrated the use of instrumentation in the evaluatic- C;f school 
based programs. During the 1970's there was considerable growth of 
evaluation methodology in the affective domain. Recently the Center 
for the Study of Evaluation at UCLA published an evaluation kit which 
includes a text on measuring attitudes (Henerson, Morris, and Fitz-Gibbon, 
1978). There is no dearth of research literature incorporating attitudinal 
assessment at the present time. There is, however, a void at the local 
level regarding development of instrumentation or selection of instrumen- 
tation* for evaluation use. The focus of this paper is assessment in 
the affective domain. The topic will be delineated sequentially and 
culminated by presentation of an illustrative model 

Background Information 

The science of assessment is complex, complicating, multifarious, 
perplexing, and inextricable; yet, it demands focus and order. Accord- 
ing to Jackson and Messick (1967), "the process of assessment involves 
the appraisal or the level of magnitude of some attribute.'' Although 



educators have come to dichotomize the cognitive and affective domains, 
in reality both domains overlap or are intertwined. However, opera- 
tionally, researchers and evaluators must focus upon specific subsets 
or domains in order to describe a phenomenon, attribute, characteristic 
or event. One branch of the affective domain contains categories such 
as values, emotions, and perceptions (Gephart, Ingle, Marshall, 1976). 
Program evaluations incorporating assessment of the affective domain 
should zero-in on one of the subcategories for measurement purposes. 

Many writers commenting on the history of attitude measurement 
agree that there was little research prior to 1920 (Jackson and Messick, 
1967; Show and Wright, 1967; Nunnally, 1967). In 1928 L. L. Thurstone 
published an article demonstrating that attitudes could be measured. 
It was 1932 when Rensis Likert proposed a method of summated ratings 
whereby respondents indicate agreement or disagreement by choosing 
response alternatives. Response alternatives were given weights under 
the Likert model. This method became quite popular and was labeled the 
Likert scale. 

According to Shaw and Wright, there is common agreement among vary-- 
ing definitions of "attitude" on one characteristic: "Attitude entails 
an existing predisposition to respond to social objects, which, in inter- 
action with situational and other dispositional variables guides and 
directs the overt behavior of the individual." (Shaw and Wright, 1967), 
In addition to this agreed upon characteristic other writers have proposed 
the following characteristics of attitudes (Hovland et al., 1953; Krech 
et al., 1962; Sherif and Cantril, 1945; Sherif and Sherif, 1956): 

1. Attitudes are based upon evaluative concepts regarding 
characteristics of the referent object and give rise 
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to motivated behavior. 

2. Attitudes are construed as varying in quality and intensity 
(or strength) on a continuum from positive through neutral 
to negative. 

3. Attitudes are learned, rather than be-fng innate or a result 
of constitutional development and maturation. 

4. Attitudes have specific referents, or specific classes 
thereof. 

5. Attitudes possess varying degrees of interrelatedness to one 
another. 

6. Attitudes are relatively stable and enduring. 

Using the above characteristics a researcher or evaluator can expect 
individuals to respond to scales incorporating negative to positive attri 
butes of a concept. 

Scale Development 

There are two prominent references covering the topic of attitude 
scale construction. The most widely known is Edwards' Techniques of 
Attitude Scale Construction (1957) and the next is Shaw and Wright's 
Scales for the Measurement of Attitudes (1967). Both texts provide 
considerable detail regarding the technical aspects of scale construction 

There are numerous methods for scale development including: equal- 
appearing intervals, gr^aded dichotomies, summated ratings, scalogram 
analysis, scale discrimination technique, unfolding technique, and 
latent structure analysis. This paper will focus upon one of the more 
common approaches used in educational evaluation methodology, i.e., 
summated ratings or Likert scale. The Likert scale or agreement scale 
technique cons-^'sts of a series of statements with extremes, favorable to 
unfavorable, or strongly agree to strongly disagree. 



According to Henerson et al. there are ten steps to the develop- 
ment of a Likert scale: 

1. Accumulate a large number of clearly favorable or clearly 
unfavorable statements about the attitude you wish" to 
measure. 

2. Ask a pilot group to respond to these statements. 

3. Score responses by assigning them from one to five points - 
five for most favorable, one for least favorable. 

4. Compute a score for each respondent by totaling the points 
corresponding to his or her responses. 

5. Identify high scorers (top 25%) and low scorers (lowest 25%). 

6. Analyze each statement according to how high and low scorers 
responded to it (item analysis). 

7.. Retain those items (approximately 20) which provided good 
discrimination between high and low scorers. 

8. Construct the scale by listing the retained statements in 
random order. 

9. Administer the scale. 

10. Compute a score for each respondent by totaling the scores 
corresponding to his or her responses. 

Evaluators must be cautioned to remember that any interpretation 
of Likert scores must be in relation to scores of others in the sample. 
Also, the Likert scale should be standardized on a representative sample 
from the target population. Given the individual differences from tribe 
to tribe, additional research should be directed at standardization with- 
in tribes as well as across tribes for scales used with the Native 
American population. 

Reliability and Validity Considerations 

Any scale must address the issue of reliability and validity. In 
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its simplest form reliability answers the question of consistent responses 
to the same items over time disregarding, of course, the subject change 
in attitude to the item{s). According to Shaw and Wright there are 
three general approaches to estimation of reliability: 

1. The correlation (r) between scores on the same test given 
at different times (test-retest method); 

2. the correlation (r) between two comparable forms of the 
same scale (the equivalent- forms method); 

3. and the correlation between comparable parts of the same 
scale (split-half method). 

The scale is considered valid if it measures what it is intended to 
measure. Shaw and Wright outline four approaches to validity: Predictive 
validity, concurrent validity, content validity, and construct validity. 

Content and construct validity are often applied to attitude scale 
instruments. With content validity it is necessary to judge the content 
of each item to the attitude domain or subscale doma'n. Also, the entire 
set of items must be examined in relation to the attitude domain being 
sampled. The reliability index may be used as an indication of construct 
validity. If the intervention produces the predicted effect and the 
changes in attitude are present, one can generalize the construct validity. 

Administration of the Scale 

Once the evaluator has a reliable and valid scale consideration must 
be given to administration procedures. Since the scale is to be used with 
groups, the evaluator must determine the unique characteristics of the 
target group and plan accordingly. One of the most important features of 
scale administration is standardization. The scale . must be administered 
in uniform format to all groups. Special considerations such as reading 



ability and length must be addressed. The directions for the test taker 
and test administrator must be clear and simply stated. The test taker 
must know how to select an answer and mark it accordingly. Sample items 
should be included in the scale. 

Psychometric Analysis and Scoring 

The evaluator must provide for objective scoring techniques. The 
agreement between scorers should be perfect. Differences between scores - 
must reflect differences in test takers not test scorers I A scoring key 
or mask should be used. 

Psychometric data should be prepared and included in the scale 
administration manual. The most important characteristic is validity. 
Evidence of validity should be specified. Reliability estimates should 
be included along with the method used. Finally any normative data or 
pilot group test resul ;s should be included in the administration manual. 
Appendices A and B provide illustration of a scale developed to examine 
attitudes toward Indians. Readers are cautioned that the appended scale 
is not for use and is developmental only at present . , The scale utilized 
a compressed Likert approach for use with primary grade children. 
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ATTITUPn TOWARD INDIANNUSS 

Attitude Towarcl Indianncss (ATI Form I - NI) is a questionnaire 
dcsicncd for elementary school aco children. It examines thrco aspects 
of children's attitudes tov/ard Indianncss: 1) Indian culture; 2) Indian 
interns; and 5) Indianncss • 

The entire questionnaire contains thirty-one items and is set uji so 
that even ncn-readors can respond to it. It Is pi*oup-adi.uni5tercd and 
takes approxir.ately 33 ninutes. The directions below for Gcl.iinistcrin^ 
the questicnnaire should be followed explicitly. Directions which should 
be read v:ord for v/ord to the pupils arc printed in large bold type, v;hiie 
intervening instructions for the teacher or other person fid.?.ini5terinfj the 
questionnaire are in italics. Plo?3e stixdy all direction s thoroughly 
before adrd )tisi:erin cy tiro quostionn.'iiir e so that you can read tiiem out loud 
in a natural and unstrcfined manner. 

Jn pTcrpavation fov the qiienti onnaire draw boxes on 

the ohalJtboard like thio: | ? \ ^^^^ pr^int 

the word ''Name*' on the beard in largo lettCTO. Make suve 
that the children* e seats ax'^e spread out in the claosroom 
and that each child has a sharp pencil. Then say: 

Today you are going to fill out a questionnaire about how you feel. 

Oefore we begin, let^s look at the board. 

Point to the boxes you have drawn on the board. 
You will sec some boxes that look like this on your questionnaire. 
Do you know the word in the first box? IVhat is it? 

Have the pupils cay the word ^'yes'' out loud. 
\ihat is the word in the last box? 

Have the pupils eay "no" out loud. 

Vihat is this sign in the middle? It is a question mark and it means 

MI don'f knov/." Now, i win. give you each a questionnaire, but do not 

make any marka on it until I tell you. 

Place a questionnaire, face i4p^ in front of each 
ohild xoith the "ilcvne" blank at the top of the 
page for him/lior. As you distribute the questionnaire » 
raiird students not to mark on them. After each 
student has a paper, say: 
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At the top of your paper, find tho word "NamoJ* It looks liko this; 

Point to the voi'd "Name'' on the board. 

Point to the ••Nsu'ne" on your paper. On the line beside tho word, print 

your name. I will help you if you need me. 

Circulate around the room to make cure the otl'dyitf^ are 
putting their ncvnes in the right place and help thoee who 
need it. If the otiidcnta do not knc^w how to write their 
lact y-icsneoj you v:ay fill th^se in for them after the 
clacQ session. If most students do not know how to ^ 
write even their first na:ries, have the name filled in 
before the class session and distribute to each 
student his/her own questionnaire. You may then skip 
the directions tliat deal with the ncrme and go directly 
to the sample questions. After students have their 
names on their papers^ say: 

Poing to the box with one(l) star. 

Check to see that all the students ca^<^ pointing to the 
star. 

I will read the words beside the star, and you listen caretully. TIi^ 
words say, ''I like to play with wy friends." Is this true for you? If 
you like to play with your friends, nark an "x" on tne t^ord "yes- next 
to the box, 

tlark an "x" on the "yes" on the chalkboard. 
If you are not sure if you like to play with your friends, mark an "x" 
on the question mark in tllo box. 

VemoYistrate on the chalkboards 

Circulate around the room and check to see if students 
are responding correctly. Expccially check any "no" 
responses. Repeat for the second item, "I think ice 
cream tastes bad. " 

Point to the two stars. Tho words next to the two star«> say," I think 

ice cream tastes bad/* If you think ice cream tastes bad, put an "x'* 

on "yes". If you do no£ think ice croom tastes bad, mark "no^t If you 



are not sure if ice cream tastes bad, put an "x*' on the question r.ark. 
Now you know how to work the questionnaire. I will read the words 
while you listen carefully. If the words are true for you , nark ''yes/' 
If the \/ords are not true foy you, mark "no." If' you are not sure about 
how you feci, put an ••x*' on the question wark. Do not look at anyone 
else's paper. Remember, put an "x" in the box that tells how you feel 
Are you ready to go on? 

Pause and check for questions. 
Find the box with three stars. The words next to the three stars say, 
••I think all Indians do beadv;ork." If you think all Indians do bcadwork, 
put on "x" on "yes*'; if you think all Indians do not do beadwork, put an 
"x" on "no". If you are not sure if all Indians do bcadwork, put an "x" 
on the question mark. Be sure to put an *'x" on the box that sho\;s how 
you feci. 

Clieck to see if all students ax'e finished before going on. 

Point to the box with four stars. The words next to the four stars say, 

"I think all Indians live in teepees." If you think all Indians live in 

teepees, put an "x" on the "yes"; if you do not think all Indians live in 

teepees, put an "x" on the 'W; if you are not sure if all Indians live 

in teepees, put an "x" on the question mark. 

Repeat these directions for all items in the questionnaire. 
Always check to be sure all students are at the sane place. 
Circulate around the room and visually check students 
responding. When you gat to item number nine^ stop and 
clarify teacher-helper. Do this by saying. 

Miss or Mr. is the 

teachers-helper. (Obtain the name of the teacher-helper ^ 
Indian Student Intern^ from the classroom teacher prior 
to the adninistration of the quest ionnaire. ) Foi^ each item 
dealing with teacher ^helper ^ read the item exactly a>id then 
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atcu'if tj^hy saying ^ Mice or 

Mr. . 

If fatigue appcax's to be prcoentj atop a>id give the 
Btudents a short rect. }/hen you finish item nicnber 
■ thirty-one^ say: 

Now you arc finished with your questionnaire. Tut your pencils do\>m, 
and lUl collect your pr.pers. 

Collect a paper from each student individually, chookiy\g 
to be sure his/her nme is on it. After the session^ go 
through and insert the student^ a last name if necesccrrj. 
Also place the grade and school on each auestiony\aire . 
For older students^ you may have them place this iKfcrr^aticn 
on it while they ai^c filling out the name section. 
Please return all used and left-over questionnaires 
to the DjrVision of Educational Research and Services. 
Thank you. 
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Nome 



Date 

School 




I think ice creom tastes M 



Yes 



Yes 



I - 



y\ <\ |rs 



I. Ithinkalllndionsdobeodworlc 



i!. 

i! 



ii 



2 I thinkoll Indians live in teepees. 



Yes 



i! Yes 



11 



AAA 

M M ^-^ 



3. All Indians ride horses. 



i Yes 
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2 



i' t> • ! 


« 
« 






j 4' I like to leorn oboul Indians, 
i 


1 Yes 


? 

« 


No 

1 

, 1 
I 

; \ 
2 1 

^^i 


j 5. . I on Qfroid of Indians. 






: 1 
; ; 

' i 
« 


Yes 


? 

• 


No ji 

; i 

!| 
!i 


I would lite to learn Indian donees. 


1 • 

\ Yes 

1 


? 

• 


:i 

; 1 

j! 

' No : 

i! 

; 1 

! . 

' i 


1 ■ 7. I would like to leorn on Indian longuoga. 

i 

,1 

:i 


Yes 


? 

f 


. i 

■ ii 

No ■! 

J i 
' 1 

• .,1 


^ ■■p- - — — . _ — 

; A A/. 

1 ^ H H i'v^ J:^ 

& I would lite the President of the 


Yes 


? 

f 


1 1 

No i 


United States to t)e an Indian. 

i 
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A • ' i 

■ 9. Myteocher-helpertellsmealolabout | 
Indians ond how they liva 


1 ' " 
Yes 


? 

1 


I 

No 1 

i 

i 
1 


Id I like to work by myself and not with 
■ the teacher-helper. 

i 


Yes 


? 

• 


i 

! 

No i 

1 

i 


II. I would like my teacher-helper to 
come back next year. 

! 


Yes 


? 

i 


. No ; 

t 
t 


i H H H A 

12 I would like to have a job like my 
teacher-helper. 


Yes 


? 

■ 


1 

No 


j 

^ l'^ 

13. I like school more when my teacher- 
helper is helping ma 

i 


Yes 


? 

9 


No 
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j 11 I think Indians con be good teachers. 

• 


Yes 


? 

1 


■ ;l 

■ '1 

.1 

No' . 

1 

1 

k 

1 

1 


15. . I lil(e the teocher-helper to help ma 


Yes 


? 

• 


1 

t 

J 

No 

1 

! 

I, 


j Id I like school more when my teacher-helper, 

is hera 


Yes 


? 

• 


1 

i 

1 

i 


1 i--^ K K A 

17. Ilikemyteacher-heiper. 


Yes 


? 

1 


1 

I 
I 

1 

] 


I K K H U K 

ii My teacher-helper is on Indiaa 

fl 


Yes 


? 


:| 

No 

! 
1 

1 

( 
1 

i) 
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j 19. IwouldliketoDeonlndiaa 

1 


Yes 


0 

f 

• 


-.1 
1 

• 1 
i 

I 

Ma t 

■I 

1 

i 

4 
t 

1 


2(1 I think oil Indians look the soma 


Yes 


r 


\ 
{ 

m 


AAA 

21 iQinnoppy. 


Yes 


9 

• 


.! 
h 

NO 

i 

■ 


11 I think Inoions nave funny names. 


: Yes 


f 

• 


ii 

■ 11 

( 

- — F 


4 

1 >^ i' w -i 

i 23 I would' like to hove an Indian for mv 

best friend 


■ Yes 

1 Vv 


? 


1 

i 

{ 

i 

No ! 
1 

! 

! 



24 I think it's oil right for men to wear 
broids. 



^ 25. I like going to school with Indians. 



26i I think Indions ore good peopla 



H. There ore Indions in my closi 



' H H i^-^ 

2a All Indians ore lozy, 



7 



• — — " 

1 ^ 

j 29. Mans are like other pcopla 


Yos 


? 

• 


4 

1 

' I 

1 

No 

* 

f 

r 

» 


3Q lihinklndionsorepet 


Yes 


? 

• 


) 

! 

•j 
1 

No i 

I 

t 
1 

f 

f 


j 31. Indians liKe being Mians 

' i 

! 


j 1 


? 
• 


1 

I 

No \ 

j 

-i 
1 

# 



I 
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